apache chokes after 300 connections
- by john titus
We have an apache webserver in front of Tomcat hosted on EC2, instance type is extra large with 34GB memory.
Our application deals with lot of external webservices and we have a very lousy external webservice which takes almost 300 seconds to respond to requests during peak hours.
During peak hours the server chokes at just about 300 httpd processes. ps -ef | grep httpd | wc -l =300
I have googled and found numerous suggestions but nothing seems to work.. following are some configuration i have done which are directly taken from online resources.
I have increased the limits of max connection and max clients in both apache and tomcat. here are the configuration details:
//apache
<IfModule prefork.c>
StartServers 100
MinSpareServers 10
MaxSpareServers 10
ServerLimit 50000
MaxClients 50000
MaxRequestsPerChild 2000
</IfModule>
//tomcat
<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="600000"
redirectPort="8443"
enableLookups="false" maxThreads="1500"
compressableMimeType="text/html,text/xml,text/plain,text/css,application/x-javascript,text/vnd.wap.wml,text/vnd.wap.wmlscript,application/xhtml+xml,application/xml-dtd,application/xslt+xml"
compression="on"/>
//Sysctl.conf
net.ipv4.tcp_tw_reuse=1
net.ipv4.tcp_tw_recycle=1
fs.file-max = 5049800
vm.min_free_kbytes = 204800
vm.page-cluster = 20
vm.swappiness = 90
net.ipv4.tcp_rfc1337=1
net.ipv4.tcp_max_orphans = 65536
net.ipv4.ip_local_port_range = 5000 65000
net.core.somaxconn = 1024
I have been trying numerous suggestions but in vain.. how to fix this? I'm sure m2xlarge server should serve more requests than 300, probably i might be going wrong with my configuration..
The server chokes only during peak hours and when there are 300 concurrent requests waiting for the [300 second delayed] webservice to respond.
Please help..
